Observer Theory, Bayes Theory, and Psychophysics

1. INTRODUCTION The search is on for a general theory of perception. As the papers in this volume indicate, many perceptual researchers now seek a conceptual framework and a general formalism to help them solve specific problems. One candidate framework is " observer theory " 1989a). This paper discusses observer theory, gives a sympathetic analysis of its candidacy , describes its relationship to standard Bayesian analysis, and uses it to develop a new account of the relationship between computational theories and psychophysical data. Observer theory provides powerful tools for the perceptual theorist, psychophysicist, and philosopher. For the theorist it provides (1) a clean distinction between competence and performance, (2) clear goals and techniques for solving specific problems, and (3) a canon-ical format for presenting and analyzing proposed solutions. For the psychophysicist it provides techniques for assessing the psychological plausibility of theoretical solutions in the light of psychophysical data. And for the philosopher it provides conceptual tools for investigating the relationship of sensory experience to the material world. Observer theory relates to Bayesian approaches as follows. In Bayesian approaches to vision one is given an image (or small collection of images), and a central goal is to compute the probability of various scene interpretations for that image (or small collection of images). That is, a central goal is to compute a conditional probability measure, called the " posterior distribution, " which can be written P (Scene | Image) or, more briefly, It provides a powerful approach to understanding and modeling human perceptual capacities. But it has a well-known limitation. For real vision problems the collection of images that might be obtained is very large. (For instance, there are about 10 15 possible true-color images of 1024 by 1024 pixel Bennett et al. Observer Theory and Bayes Theory resolution.) Therefore P (I) and P (I | S) are either 0 or near 0 for most images and the form of Bayes rule given above is either undefined or unstable. We cannot remove this problem by conditioning on large collections rather than on small collections of images, because the task we typically face in image understanding is to interpret a given single image or small collection. In special cases the instability problem can be avoided by the Poggio and Girosi, 1989). What we need, however, is a general form of Bayes rule that allows conditioning on events of probability zero and that requires …